This issuboptimal because the classifier isinherently biased towards a subset of the modalities. To alleviate this shortcoming, we propose a novel regularization term based on the functional entropy.

Many recent datasets contain a variety of different data modalities, for instance, image, question, and answer data in visual question answering (VQA).

Backpropagation, which uses the chain rule, is the de-facto standard algorithm for optimizing neural networks nowadays. Recently, Hinton (2022) proposed the forward-forward algorithm, a promising alternative that optimizes neural nets layer-by-layer, without propagating gradients throughout the network. Although such an approach has several advantages over back-propagation and shows promising results, the fact that each layer is being trained independently limits the optimization process. Specifically, it prevents the network's layers from collaborating to learn complex and rich features. In this work, we study layer collaboration in the forward-forward algorithm. We show that the current version of the forward-forward algorithm is suboptimal when considering information flow in the network, resulting in a lack of collaboration between layers of the network. We propose an improved version that supports layer collaboration to better utilize the network structure, while not requiring any additional assumptions or computations. We empirically demonstrate the efficacy of the proposed version when considering both information flow and objective metrics. Additionally, we provide a theoretical motivation for the proposed method, inspired by functional entropy theory.

In this paper, we investigate how to measure the intelligence of systems under specific structures. Two indicators are adopted to characterize the intelligence of a given structure, namely the function diversity of the structure, and the ability to generate order under specific environments. A measure of intelligence degree is proposed, with which the intelligence degree of several basic structures is calculated. It is shown that some structures are indeed "smarter" than the others under the proposed measure. The results add a possible way of revealing the evolution mechanism of natural life and constructing life-like structures with high intelligence degree.